A Brief Description of the Levenberg-Marquardt Algorithm Implemened by levmar

The Levenberg-Marquardt (LM) algorithm is an iterative technique that locates the minimum of a function that is expressed as the sum of squares of nonlinear functions. It has become a standard technique for nonlinear least-squares problems and can be thought of as a combination of steepest descent and the Gauss-Newton method. This document briefly describes the mathematics behind levmar, a free LM C/C++ implementation that can be found at http://www.ics.forth.gr/ ̃lourakis/levmar. Introduction The Levenberg-Marquardt (LM) algorithm is an iterative technique that locates the minimum of a multivariate function that is expressed as the sum of squares of non-linear real-valued functions [4, 6]. It has become a standard technique for non-linear least-squares problems [7], widely adopted in a broad spectrum of disciplines. LM can be thought of as a combination of steepest descent and the Gauss-Newton method. When the current solution is far from the correct one, the algorithm behaves like a steepest descent method: slow, but guaranteed to 1 converge. When the current solution is close to the correct solution, it becomes a Gauss-Newton method. Next, a short description of the LM algorithm based on the material in [5] is supplied. Note, however, that a detailed analysis of the LM algorithm is beyond the scope of this report and the interested reader is referred to [5, 8, 9, 2, 10] for more comprehensive treatments. The Levenberg-Marquardt Algorithm In the following, vectors and arrays appear in boldface and is used to denote transposition. Also, and denote the 2 and infinity norms respectively. Let be an assumed functional relation which maps a parameter vector to an estimated measurement vector . An initial parameter estimate and a measured vector are provided and it is desired to find the vector that best satisfies the functional relation , i.e. minimizes the squared distance ! ! with ! " # . The basis of the LM algorithm is a linear approximation to in the neighborhood of . For a small %$'&( , a Taylor series expansion leads to the approximation